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Abstract 


We consider the critical p-Laplacian system 



X G O, (0.1) 


where Ap := div(|Vu|P ^Vm) is the p-Laplacian operator defined on := 

{u G LP (R'^) : |Vm| G LP(R^)}, endowed with norm ||m||di.p := 



isfy a + b = p,a + P = p* ■= 77 ^^^ the critical Sobolev exponent, fl is R^ or 

a bounded domain in R-^, dI’^{Q) is the closure of (7^(11) in Under 

suitable assumptions, we establish the existence and nonexistence of a positive least 
energy solution of dO.ll ). We also consider the existence and multiplicity of nontrivial 
nonnegative solutions. 
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1 Introduction 


Equations and systems involving the p-Laplacian operator have been extensively stud¬ 
ied in the recent years (see, e.g., E[3]|5]0[ 8 ]|9][TOl[13][161 [iTl[19]|20l|22]|23] and 
their references). In the present paper, we study the critical p-Laplacian system 

f—ApU — ^\u\°‘~'^u\v\^ = ~'^u + a; G fl, 

< -ApU - y -I-cc G fl, (1-1) 

[u,t;in 


where ApU := div(| Vu|^ ^Vu) is the p-Laplacian operator defined on := 

{u G LP (M.^) : |Vm| G ivP(]R'^)}, endowed with norm ||u||£| 1 ,p := |Vrt|*’da;) ^, 

A > 3, 1 < p < N, fj,i, fj ,2 > 0, 7 7 ^ 0, a, 6 , a, /? > 1 satisfy a-|- 6 =p, q;-|-^ = 
P* •= critical Sobolev exponent, fl is or a bounded domain in K^, and 

DQ’^{n) is the closure of C(j”(r2) in Note that we allow the powers in the 

coupling terms to be unequal. We consider the two cases 

(Hi) = A = 0,/ti,/r2>0; 

(H 2 ) fl is a bounded domain in R^, A > 0, /ii, /X 2 = 0 ,7 = 1. 

Let 

fo \Vu\Pdx 

S:= inf ^-- ( 1 . 2 ) 

be the sharp constant of imbedding for ^ LP (17) (see, e.g., fl]). Then S 

is independent of 17 and is attained only when 17 = R^. In this case a minimizer 
u G D^’P{M.^) satisfies the critical p-Laplacian equation 

— ApU = |u|^ ~^u, X G R^. (1.3) 

Damascelli et al. IH recently showed that all solutions of (11.3b are radial and radially 
decreasing about some point in R^ when 1 < p < 2 < p*. Sciunzi iH] extended this 
result to the case 2 < p < A. By exploiting the classification results in |[4][T8l, we see 
that, for 1 < p < A, all positive solutions of ( 11.3b are of the form 


U, 


and 




p- 


£P-1 


N —p 


ep-1 +\x — y\'p 


f) ' 


/B'V 


\yU,,y\PdX= \U,,y\P dX = ST . 


e > 0, y G R^, 

(1.4) 

(1.5) 


/b« 
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In the case {Hi), the energy functional associated with the system (II. lb is given by 


I{u,v) = - [ {\yu\P+ \Vv\P) -[ (/ii|u|P*+/i2|u|P*+7|u|“|t;|'^), 

P Jrn P Jrn 

{u,v) € D, (1.6) 

where D := D^’P{E.^) x D^’P{R^), endowed with norm ||(m,u)||^ = ||rt||^i,p + 
||t;||^i,p. In this case, ( II.lb with a = /3 and p = 2 has been studied by Chen and Zou 
liniEl. Define 


M = \{u,v) e D ■.U^ 0 ,v ^ 0 , [ \yu\P = [ {fii\u\p"+ ^\u\°‘\vf), 

I- Jr^ Jr^ p 

[ \Vv\P=j {p^\vf+^\ur\v\P)]. 
Jrn Jrn p* '} 

It is easy to see that J\f ^ % and that any nontrivial solution of (II.lb is in J\f. By a 
nontrivial solution we mean a solution {u, v) such that it ^ 0 and u ^ 0. A solution is 
called a least energy solution if its energy is minimal among energies of all nontrivial 
solutions. A solution (u, v) is positive if u > 0 and u > 0, and semitrivial if it is of the 
form (u, 0) with it ^ 0, or (0,u) with u ^ 0. Set A := inf(„_p)gjv'^(w,i'), and note 
that 


A = inf — 


{\Vu\P A\Vv\P) 

if / {piWf +P2\vf +i\unyf). 

J.,v)GAf ly Jrn 


(u,' 

Consider the nonlinear system of equations 

' p*-p a-p g 

pik p H- i-K p t p = 1, 

+ (1-7) 

k > 0, I > 0. 

Our main results in this case are the following. 

_ N—p _ N—p ^ 

Theorem 1.1. 7/'(iPi)/toWsand 7 < 0, f/ie« A = ^^ ^ )S^andAis 

not attained. 

Theorem 1.2. If {Hi ) and 

{Cl) ^<p<N,a,j5>p, and 

0 < 7 < t:: -r^T mm ■ 


(3-p)2 


j-pi / g -p \ P2 

tgVS —n/ ’ d\a — p) J 


( 1 . 8 ) 


3 






(1.9) 


{C 2 ) <p < ^,a,^ <p, and 


7 > 


Np"^ (Pi (P-P2(p-a\^\ 

(iV-p )2 a ip-a/ ' p\p-p) i 


hold, then A = jj{kQ + Id)S p and A is attained by ^/ToU^^y), where 

(fco, Id) satisfies ( li.7l ) and 


ko = min{fc : (fc, 1) satisfies ( li.7l >l. (1-10) 


Theorem 1.3. Assume that < p < , a, j3 < p, and (Hi) holds. If"f > 0, then 
A is attained by some (U, V), where U and V are positive, radially symmetric, and 
decreasing. 

Theorem 1.4. (Multiplicity) Assume that < p < a, f) < p, and (iJi) holds. 
There exists 


7i G (0, 


Np^ 


( Pi / 2 — f3 \ 2 p2 — o . \^2 'y i 

I a V2-a/ ’ /3 V2-/3/ jJ 


2 -a\^' 


{N-pf 


■ max 


such that for any 7 S (0, 71 ), there exists a solution (fc( 7 ), ^ 7 )) of ( 17.71) satisfying 


l{^/H^)Ue,y,</l{^)Ue,y)> A 


and { f/k{y)Ue,y, ^/Wt)Ue,y) is a (second) positive solution ofGJ}- 
For the case {H 2 ), we have the following theorem. 

Theorem 1.5. If{H 2 ) holds, p < s/N, and 

0 < A < -^Ai(f7), 

(a“ 6 ^)p 

where Ai(r2) > 0 is the first Dirichlet eigenvalue of—Ap in then the system ( 17.71 ) 
has a nontrivial nonnegative solution. 


2 Proof of Theorem ll.il 

Lemma 2.1. Assume that {Hi) holds and —00 < "f < 0. If A is attained by a couple 
(u, v) € M, then {u, v) is a critical point of I, i.e., {u, v) is a solution 0/(0- 

Proof. Define 

Afi := ^{u,v) e D : u ^ 0, v ^ 0, 

Gi{u,v) := [ |Vur - / {Pi\nf + = o|, 

Af 2 ■= ^{u,v) e D : u ^ 0, V ^ 0, 

G2{u,v):=[ \Vv\P-[ {p2\v\p* + ^\u\°‘\vf) = o\. 

7r«- Jrn p* ' } 
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Obviously, J\f = J\fi fl A/ 2 . Suppose that {u,v) G A/” is a minimizer for I restricted 
to J\f. It follows from the standard minimization theory that there exist two Lagrange 
multipliers Li, L 2 G R such that 

/'(m, v) + LiG'i{u, v) + L 2 G 2 {u, v) = 0. 


Noticing that 

I' (u,v)(u,0) = Gi(u,v) = 0, 
l'{u,v){0,v) = G 2 {u,v) = 0, 

G[{u,v){u,0) =-{p* - p) f pilulP’+ {p - a) f ^\u\°‘\vf, 

JR^ JR’^ P 

G[{u,v){0,v) = -l3 f > 0, 

p* 

G2{u,v){u,0) = -a f > 0, 

Jr!^ P 

G2iu,v){0,v) = -{p*-p) f P 2 \v\p''+ {p - p) [ 

JRN jR" P 

we get that 

f G[ {u, v) {u, 0)Li + G 2 (u, u) (m, 0)^2 = 0, 

[G[{u,v){0,v)Li + G2iu,v){0,v)L2 = 0, 

and 

G[{u,v){u,0)+ G[{u,v){0,v) =-{p*-p) f |Vm|^ < 0, 

JR" 

G'2{u,v){u,0) + G'2iu,v){0,v) = -{p*-p) [ |Vu|P < 0. 

Jr" 

We claim that |Vm|p > 0. Indeed, if |Vu|p = 0, then by (11.2b . we have 

« P* 

/jjjv 1^1^ < S~~ ^ Jjjjv ^ = 0. Thus, a desired contradiction comes out, 

u = 0 almost everywhere in R^. Similarly, | Vu|^ > 0. Hence, 

|Gi(u,-i;)(u,0)| = -G[{u,v)(u,0) > G[{u,v){0,v), 

|G2(^^:t^)(0,1')| = -G2 (m,w)(0,u) > G2(m,u)(m,0). 

Dehne the matrix 


{ G[{u,v){u,0) G2 {u,v){u,0) \ 

V G^(u,z;)(0,u) y’ 


then, 

det(A/) =|Gi(u,u)(u,0)| • |G 2 (m, u)(0, u)! 

- G[{u,v){0,v) ■ G2(m,u)(m,0) > 0, 


which means that Li = L 2 = 0. This completes the proof. 


□ 
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Proof of Theorem II.1[ It is standard to see that A > 0. By (11.4b . we know that 

p-N 

■= Mi” Ui^o satisfies —ApU = p,i\u\P ~'^u in , where i = 1,2. Set ei = 
(1,0,--- ,0) € and 

{uR{x),vn{x)) = {ujf,^{x),ujp,,{x + Rei)), 

where i? is a positive number. Then, u/j ^ 0 weakly in (R^) and u/j ^ 0 weakly 
in LP (R-^) as i? —>■ +oo. Hence, 


lim 

R—^-\-cx> 


u'^v^dx = lim 


R^-\-oo J^N 


a P*(/ 3 -l) 


URVh ^R 


< lim ( [ Mp ^Uijdx) ” ^ ( [ Up dec) ” 

-«^+ooVV « " ; Wb« ^ 


= 0 . 


Therefore, for R > 0 sufficiently large, the system 

Jkn iVuflJPdec = /jj;,, /iiu^ dec 

^ =f^’’ piu^dx + /rjv ^u^u^dae, 

Jrn \yvR\Pdx = /jj„ dx 

— ^R^ /rw M 2 Ur dx + /jjjv f?tx^u^dx 

has a solution {Ir, sr) with 

lim {\tR - 1| + |sR - 1|) = 0. 

R —^-j-oo 


Furthermore, {^^/IrUr, ^/^vr) G N. Then, by (11.5b . we obtain that 


A= inf I{u,v) < I{0 ]^ur, ^^/J^vr) 

{u,v)GAf 


1 

N 


(tR [ iVu/jl^da; + SR [ iVuRl^dx) 

^ jRN JRW / 


1 N — p N-p ^ 

= —{tRp^ ” +SRP2 ” )S~ 

_ N-p _ N-p ^ 

which implies that ^ ^ ^ + M 2 ” )S~. 

For any {u, v) G Af, 


Jr^ 


|Vu|^da; < fii [ \u\p dx < piS [ iVul^dx) ” . 

JR^ ^Jr^ ' 


N-p 


N—p 


Therefore,/jjjv I Vu I ^dx >/i;^ ” S'p . Similarly, |Vu|^dx > ^^2 ^ S p . Then, 

N-p N-p ^ 

^ > ;^(Mi ” + M 2 ” )S'p. Hence, 


1 ^-P N-p ^ 

^ = j^iPi ” +M 2 ” )S~- 


( 2 . 1 ) 
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Suppose by contradiction that A is attained by some (u, v) € J\f. Then (|m|, |r;|) € 
A/” and I{\u\, |t;|) = A. By Lemma ITTl we see that (|it|, |t;|) is a nontrivial solution 
of (fTTIi. By strong maximum principle, we may assume that u > 0 ,u > 0 , and so 
u“u^da; > 0 . Then, 


/R" 


|Vu|^da; < pi / |u|^ dx < fiiS ^ f / iVwl^dx) ” , 


which yields that | Vul^dx > ^ S p . Similarly, |Vu|Pdx > ^2 ’’ S p . 

Therefore, 


1 /■ 1 w-p iV-P „ 

A = I{u,v) = — (|Vm|p+|V u|P)dx > —(^1 ^ )S'^, 

iV JjjN 

which contradicts to ( 12 . 11 ) . This completes the proof. 


□ 


3 Proof of Theorem 11.21 

Proposition 3.1. Assume that c, d G K satisfy 


Mic p + f?c p dp > 1, 

P —P o a 0 — p 

/i2d p + ^cp d p > 1, 
c > 0, d > 0. 


( 3 . 1 ) 


// Y < p < N,a,l3 > p and ([T^ holds, then c + d > fc + /, where fc, / G K. satisfy 

(O- 

Proof Let y = c + d,x = 2 , Vo = k + l, and xq = j. By (13.11 ) and ( 11.71 ). we have that 

p*-p (x + 1) p 


y p > 


Pix p + -^x p 
p1^ (x + 1)^ 

y ^ := f2{x), 

P2 + ^a;p 


— :=/i(x), t/o ” =/i(xo), 


do = /2(a:o)- 


Thus, 


/l(a^) = 


f2{x) = 


p — 2p a — 2p 

Q:7(x + 1 ) p X p 


pp*(/rix'” p + ^x p^)2 


P*{P*-P)P1 £ , a / ^ 

-X p + px — (a — pj 


/37(x + 1) p 


PP*{P2 + f?a;p)2 L 


. , - ELZP p*{p* -P)P2 

[p — pjx p — ax p + ■ 
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Let and 


gi{x) = - ?^xp + Px- [a-p), 


a7 


( \ (o \ ^ ^ p*{p* -p)p2 

92 {x) = [p — p)x p — ax p + ■ 


Pp 


It follows from (II. 8 I 1 that 


max gi[x) = 

tc€(0,+oo) 


(x) =gi{xi) =(/3-p)( - (a-p) < 0 , 


min g 2 {x) = g 2 {x 2 ) = - p(^—-) ” + 

xG(0,+oo) \/j — Z?/ 


p*(p* -p)Pi, 

-P\^ , P*{P* -P)P2 


P-P' 


Pi 


> 0 . 


That is, fi{x) is strictly decreasing in (0,+c») and f 2 {x) is strictly increasing in 
(0, + 00 ). Hence, 


y” p" > max{/i(a:), f 2 {x)} > min ( max{/i(x), / 2 (a;)} ) 

xG(0,+oo) \ / 

(^ma.x{fi{x), f 2 {x)}^ = 2/0 


= min 


p —p 
_ P 


where {/i = 72 } := {x € (0, + 00 ) : fi{x) = f 2 {x)}. This completes the proof. □ 

Remark 3.1. From the proof of Proposition \3. 1 1 it is easy to see that the system ( 113 , 
under the assumption of Proposition 1X71 has only one real solution {k, 1) = (fco, ^o), 
where (fco, ^o) ^ defined as in M.lOh 

Define functions: 

Fi(fc, 1) := pik"^ + ‘^k^lp - 1, fc > 0, / > 0; 

p* 


F2{k,l) := 2 ^ 2 ^*" p” + —kpl^p” — 1 , > 0 ,/ > 0 ; 

P* 


m := 




^ p- 


sa7^ 


' (l — pik^ p”)^, 0 < k < 


(3.2) 


•= (1 - P 2 ^'’ 7“’ 0<1<P2 

Then, /(fc)) = 0 and F 2 (fc(/), /) = 0. 

Lemma 3.1. Assume that < p < P < p,^ > 0. Then 
Fi{k,l) = 0, F2ik,l) =0, k,l>0 
has a solution (feg, hf) such that 


(3.3) 


F 2 {k,l{k)) < 0, Vfce(0, fco), 


(3.4) 










that is, (fcg, lo) satisfies ( I7.70I ). Similarly, 0.31) has a solution (fci, li) such that 


that is. 


Fi(fc(0,0 <0, vze (0,/i), 


(3.5) 


(fci, h) satisfies ( 17.71 ) fln^7 h = miii{/ : (k, 1) is a solution of ( I7.7D I. (3.6) 


Proof. We only prove the existence of (kofio). It follows from Fi{k, 1) = 0, k,l > 0 
that 

l = l{k), \/kG (0,p”^). 

Substituting this into 7^2 (fcj 0 = 0’ we have 


Setting 


,R-Z2.\J < 37 , 

p^[~) ” j + —fc 

n — pi/c p j =0. 


■.ay 
Va7/ 


(P* p)(p Q;) 


/(t) ® (i - 


a7 


-(-) 

\a7/ 


(p* -p)(p-q) 


P)(p-q) / P -p \ • 

( 1 — /ii/c P J 


(3.7) 


(3.8) 


then the existence of a solution of ( I3.71 i in (0,/i]^ is equivalent to f{k) = 0 

— ^P 

possessing a solution in (0, ’’ ). Since a,fi<p, we get that 


lim /(fc) = —oo, 
fc—)-0+ 




> 0 , 


which implies that there exists ko G {O, Pi ’’ such that /(fco) = 0 and f{k) < 0 
for k G (0, fco). Let Iq = l{ko). Then (fco, Iq) is a solution of (I3.31 l and (I3.41 i holds. □ 

Remark 3.2. From < p < ^ a,P < p, we get that 2 < p* < 2p. It can be 
seen from ^ < p < N and a, fi > p that 2 < 2p < p*. 

Lemma 3.2. Assume that < p < ^ (i < p, and ( 17.91 ) holds. Let (fco, Iq) be 
the same as in Lemma \3J\ Then, 


(fco + ^ max{/ii,p 2 } < 1 


(3.9) 


and 


F 2 (fc,Z(fc)) < 0, Vfce(0,fco); Fi(fc(fc),Z) < 0, V/e(0,Zo). (3.10) 


9 








Proof. Recalling (13.2b . we obtain that 


(P*\ 

^ p f 

\ajJ 

/3V 

/pVi 

V ^ 

■VA: 

V 0(7 

j 


forfc e ((7rrf)^>Aii ^ 


* * P — 0 

p-p / _i , p -p \ 0 /p — Q; 

* ('*' ') (t^- 

)=0 ,l'(i:)>0 forte(0 ,(^) 
). From 


PiP, 
- k p 

P 



p'-p), and l'{k) < 0 


i"Ck) = 




p*-p \ ^ / _i - p*-p \ / Qi(p—Q:) - p*-p 

' 1 ., ^ b 1 — k ^ 




piP{p- 0) 


= 0 


^ ^ we have k = Then, by ([T^, we 

get that 

min ^ l'{k)= min ^ l'{k) = l'{k) 



P*(P 


* -p)Pi \ ^/ P-0 \^ 
pa'y / \p — a/ 


> - 1 . 


Therefore, Z'(A:) > —1 for k € (0,^i *’* with f { ( 2 p-p*)l,ii 3 ) which 

— P — _P 

implies that l{k) + k is strictly increasing on [O, . Noticing that k^ < , 

we have 

p p _ p /(fco) + /cq = ^0 + ^0i 

that is, ni{ko + Iq) p < 1. Similarly, ^ 2(^0 + /q) < 1- To prove (13.10b . by 

Lemma im it suffices to show that (fco, /q) = {kiji). It follows from (13.4b and (13.5b 
that ki > kf) and Iq > li. Suppose by contradiction that ki > kg. Then l{ki) + ki > 
l{ko) + ko- Hence, li + k{li) = l{ki) + ki > l{ko) + ko = lo + k{lo). Following 
the arguments in the beginning of the proof, we have I + k{l) is strictly increasing for 

— _P 

I £ [ 0 , fj ,2 . Therefore, li > Iq, which contradicts to Iq > h- Then, ki = ko, and 
similarly, Iq = li. This completes the proof. □ 


Remark 3.3. For any 7 > 0, the condition ( 17.91) always holds for the dimension N 
large enough. 

Proposition 3.2. Assume that < p < ^,a, 0 < p, and ( 17.9b holds. Then 


{ k I f. ko Iq, 

Fi{k,l)>0, F2{k,l)>0, (3.11) 

k,l>0, ik,l)f{0,0) 
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has an unique solution (fc, /) = (fco, lo)- 


Proof Obviously, (fco, ^o) satisfies (13.1 11 1. Suppose that (fc, 1) is any solution of (13.111 1. 
and without loss of generality, assume that k > 0. We claim that / > 0. In fact, if 

^ = 0, then k < ko + Iq and Fi(k, 0) = pifc p — 1 > 0. Thus, 

~ p —p p —p 

1 < Pik p <Pi{ko + lo) p , 

a contradiction with Lemma 

Suppose by contradiction that k < kg. It can be seen that k{l) is strictly increas¬ 
ing on (O, (^^)p*'-p] and strictly decreasing on [(2^)?*-? , ^2 and fc(0) = 

^(^2 ) = 0- Since 0 < fc < fco = ^{Iq), there exist 0 < < Z 2 < M 2 such 

that k{li) = k{l 2 ) = k and 

F2(k,l)<0^k<k{l)^h<l<l2. (3.12) 

It follows from L'i(fc, 1) > 0 and ^ 2(^5 0 > 0 that I > l(k) and I < li or I > 12 - By 
( 13.10b . we see F 2 (k,l(k)^ < 0. By ( 13.12b . we get that li < l(k) < Z 2 . Therefore, 
I > I 2 . 

On the other hand, set I 3 := ko + Iq — k. Then, I 3 > Iq and moreover, 

^((3) + fco + ^0 ~ ^ = ^(^3) F I3 > k{lo) F Iq = kf) + Iq, 

that is, k{l 3 ) > k. By (13.12b . we have h < I 3 < h- Since k F I < fco + Zq, we 
obtain that I < ko F Iq — k = I 3 < 12 - This contradicts to I > I 2 , which completes the 
proof. □ 


Proof of Theorem 1 1.21 Recalling (11.4b and (11.7b . we see that ^/ToUg^y) G 

J\f 13 3. nontrivial solution of (fTTT i. and 


A < I(^/%Ug^y, 0^Ug^y) = ^{koF lo)S p . 


(3.13) 


Let {(u„,u„)} C A/" be a minimizing sequence for A, i.e., I{un,Vn) —>■ A, as 
n ^ 00 . Define c„ = {unf^dx)^ and Then, 

SCn< [ \WUn\^dx= [ {pi\Un\^ + ^ |li„ | “ |?;„ |^) dcC 
7rn Jrn P 


p_ Q/'W « E 
< tlicf H- ^Cn dn , 


Sdn< [ |Vu„|^’dx= [ (m2|u„|^ + ^|ii„|“|u„|^)da; 

Jrn Jrw P 


(3.14) 


eI “ S. 

< P2df H- ^cfdn- 

p 


11 












Dividing both sides of the inequalities by Scn and Sdn, respectively, and denoting 
Cn = ° p , dn = , we deduce that 

Sp*-p Sp* ~p 

{ P*-P Q-P 

Pldn” + > 1, 

.. p*-p a . fi-P 

^^2dn^ >1, 

that is, Fi{cn,dn) > 0 and F 2 {cn,dn) > 0. Then, for ^ < p < N, a, (3 > p. 
Proposition 13.11 and Remark [TTI ensure that c„ + > k + I = fcp + Zg; for < 

p < a, (3 < p, Proposition l3.2l guarantees that c„ + c?„ > /cp + lo- Therefore, 

Cn + dn> (ko + lo)S^^ = (fcp + lo)S^. (3.15) 

Noticing that I{un,Vn) = jj J^n (|Vu„|p + |V?;„|p), by ( I3.13l l and ( 13.14b . we have 

S(Cn + dn) < NI{Un,Vn) = NA + o(l) < (fcp + lo)S~ + o(l). 

Combining this with (13.15b . we get that c„ + d„ —?► (fcp + Iq)S p as n —?► c». Thus, 

1 1 jv 

A= lim I{un,Vn)> lim —S'(c„ + d„) = —(fcp + Zp)^ p . 

n—b-oo n—b-oo J\ J\ 

Hence, 

A=^{ko + lo)S'^ =I{^U,,y, ^oUe,y)- (3.16) 

□ 


4 Proof of Theorems [L3] and 11.41 


For {Fli) holding and 7 > 0 , define 


A' 


inf liu.v), 

iu,v)GAr' 


( 4 . 1 ) 


where 

AT' := {(u, u) € \ {(0,0)} : [ (iVu^ + IVp^) 

= / {Pl\uf + P2\vf +j\u\‘^\vf)]- 

JRiV J 


( 4 . 2 ) 


It follows from M C M' that A' < A. By Sobolev inequality, we see that A' > 0 . 
Consider 


r~ApM = pi|u|P* X G B{ 0 ,R), 

l-ApV = p 2 \vf-^v+j;}\u\‘^\vf-^v, xGB{ 0 ,R), ( 4 . 3 ) 

[u,v G H^{B{ 0 ,R)), 
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where B{0,R) := {x € : \x\ < R}. Define 


M'{R) := I {u, v) G HiO, R) \ {(0,0)} : [ (| + | Vz;^) 

'■ Jb(q,r) 


Ib{o.r) 


+H2\vf +j\unvf)} 


and 


A'(R) := inf I(u,v), 
{u,v)sj\r'{R) 


(4.4) 


(4.5) 


wherei7(0,i?) := Hq{^B{0,R))xHq[B{0,R)). Fore G [0,min{a,/3}—l),consider 


—ApU = fii\u\P ^ ^ ‘^u\v\^ a:GB(0,1), 


—ApV = fi 2 \v\^* ^ ^ a;GS(0,1), 

^u,z;GiFi(^(0>l))- 


(4.6) 


Define 

Ie{u,v)-.=- [ (iV^r + |Vz;r) 

P Jb{Q,1) 

+ i\ur^\vf-^) 

P — Jb(o,i) 

\{( 0 , 0 )}:Ge(zr,z;) := 

/s(0,l) 


AC :={ {u, v) G H{0, 1) \ {(0,0)} : G,{u, v) := / (iVu^ + iVz;^) 

- / (/rilzxK-2^ + ti 2 \v\P‘-^^ + j\ur-^\vf-^) = o}, 

Jb(o.i) ’ 


and 


A ■■= , inf ^ Ie{u,v). 

{u,v)£j\f^ 


(4.7) 

(4.8) 


Lemma 4.1. Assume that < p < A-,a, fi < P- For e G (0, min{a, /3} — 1), there 


holds 


Ae<min| inf I^lu^O), inf /e(0,z;)]-. 

\{u,0)eM^ ^ (0.v)GJ\ri ’] 


{0,v)GJ\f' 


Proof. From min{a, /?} < it is easy to see that 2 < p* — 2£ < p*. Then, we may 
assume that Ui is a least energy solution of 

-Apu = u G i7o(B(0,l)), z = l,2. 

Therefore, 


/e(ui, 0) = ai := inf /e(u,0), /^(O, M 2 ) = 02 := inf 4(0, u). 

('u,0)gA4' (0 ,d)£A/’j 
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It is claimed that, for any s € M, there exists an unique t{s) > 0 such that ( ^t{s)ui, 
^t{s)su 2 ) S A/’g. In fact. 


P* -p-2e 

t{s) P = 


/b( 0.1) (|Vui|^' + |snVu2r) 




qai + qa2\s\P 


qai + qa 2 \s\P'' + |s|/3 ®7|ui|“ ^|m 2 |'’ 

where q := = ^p^- 2 ~e^+ 2 ep^ ^ N as e ^ 0. Noticing that f(0) = 1, we 

have 


t'{s) (/3-^)/B(0.1)7|wir "\U 2 \^~ 


lim -r-fR^^-o 

s-»-0 |s|P ® 


(p* - 2s)ai 


that is. 


(/3 — e) ,, 7 |mi|“ ®|u 2 |^ ^ 

t'{s) = --|s|^“^“^s(l + o(l)), ass-j^O. 


ip* - 2e)ai 


Then, 




and so. 


P*-2e /b(0 1)T'I^iI I'“2|^ / X 

f(s) p =1-^-|s|^ ®(l + o(l)), as s —>• 0. 

pai 


Since i — i = „ ^ we have 

p q p*—2s^ 


Ae <Ie{^t{s)ui, ^t{s)sU2) 

= (- - ^ ^ ) (901 + gaals^*”^'' + f 7 |Mir"^|u 2 |^"'')f^^ 

\p p* - 2e2 V 9b(o.i) > 

=ai - [ 7|ui|““1m 2|^-" + o(|s|^-") 

9^ 9b(o.i) 

<ai = inf /g(u, 0) as IsI small enough. 

{u,o)eAr;^ 

Similarly, < inf(o,p)gA/'^ ^e(0, v). This completes the proof. □ 

Noticing the definition of uj/j,- in the proof of Theorem ll.il similarly as Lemma l4n 
we obtain that 

A'< mini inf l(u,0), inf 1(0,u)]- 
I (ix.O)eA^' (0.j;)GAf' J 

= min{/(w^i,0),/(0,w^J} (4.9) 

. (-1 -PLzP N 1 N'l 

= min{—Pi Sp,—P 2 5'i>|. 
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Proposition 4.1. For any e € (0, inin{a, /3} — 1), system M.6^ has a classical positive 
least energy solution (u^, Vg), and u^-, are radially symmetric decreasing. 

Proof. It is standard to see that A, > 0 . For (u, v) G with u > 0, t; > 0, we 
denote by {u*,v*) as its Schwartz symmetrization. By the properties of Schwartz 
symmetrization and 7 > 0 , we get that 



, 1 ) 




JB( 0 , 1 ) 


Obviously, there exists f* G (0,1] such that G Aff Therefore, 

h{VPu*, </¥v*) = (-- f + \vv*\p) 

\p p — ze/ Jb(o.i) 


< 


p p* — 2e/ 
p* — 2e — p 


PiP*-2e) 7 ^( 07 ) 

= Ie(.u,v). 


B(O.l) 

(|Vu|P + |Vt;|P) 


(4.10) 


Then, we may choose a minimizing sequence (m„, Vn) G Af^ of such that (u„, u„) = 
«>o and Ig{un,Vn) —>■ Ag as n —>■ 00 . By (14.1 Oi l, we see that Un,Vn are uni¬ 
formly bounded in (i?(0,1)). Passing to a subsequence, we may assume that Un 
Ug,Vn Vg weakly in iJp (5(0,1)). Since iFg (5(0,1)) Lp*“^®( 5(0, 1)), we 

deduce that 


's(o.i) 
= lim 


'B(O.l) 


{pi\Unf + p.2\Vn\^ + l\ 


Unr-^\Vnf-^) 


p{p* - 2 e) 

—--- hm Is{Un,Vn) 

P* — Is — P n^OO 

- "'U. > 0 , 


p* — 2£ — p 

which implies that (ue, Ve) 7 ^ (0,0). Moreover, rte > 0, Ue > 0 are radially symmetric. 
Noticing that Jb(o,i) ^ liinn->oo Jb(o.i) + |V'i;„|J’), we 

get that 

f (ivu,r+ivu,r) < / {pi\Uef-^^+p2\Vef-^^+l\Uer^\v,f-^). 

Jb(o.i) Jb(o.i) 


Then, there exists G (0,1] such that (G and therefore, 
Ae < Ie{</l^Ue, </t^Ve) 


< lim 


p* — 2e — p 


n^oo p{p* - 2e) Jb{0A) 

— lim /g (Un ^ ) — A ^, 


(|Vu„r + |Vt;„r) 
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which yields that tg = 1, {ue, Ve) £ I{ue, v^) = Ag, and 


/B(O.l) 


(|Vu,|P + |Vt;,r) 


lim [ (|Vw„r + |Vt;„r). 


That is, Un —>■ Ue,Vn —>■ Ve stTongly in Hq(^B{0, 1)). It follows from the standard 
minimization theory that there exists a Lagrange multiplier L £ M satisfying 


+ LG'^{Ue,Ve) = 0 . 


Since/'(ue,'!;e)(Me,t;£) =Ge{ue,Ve) = 0 and 

G'e{Ue,Ve){Ue,Ve) 

= _(p*_2e-p) [ + 

Jb{o,i) 

we get that L = 0 and so /'(itgjUg) = 0. By = I{ue,Ve) and Lemma I tTI we 
have Me ^ 0 and ^ 0. Since Ue,Ve > 0 are radially symmetric decreasing, by 
the regularity theory and the maximum principle, we obtain that {ue, Ug) is a classical 
positive least energy solution of i4.6i . This completes the proof. □ 

Proof of Theorem ll.31 We claim that 


A'{R) = A' for AIR >0. (4.11) 

Indeed, assume i?i < i? 2 - Since Af'{Ri) C N'{R 2 ), we get that A'{R 2 ) < A'{Ri). 
On the other hand, for every (u, v) £ Af'{R 2 ), define 


{ui{x),Vi{x)) := ((^) 



then it is easy to see that {ui,vi) £ Af'{Ri). Thus, we have 

^'(^ 1 ) < y{u,v) £ A/''(i?2), 


which means that A'{Ri) < A'{R 2 ). Hence, A'{Ri) = A'{R 2 ). Obviously, A' < 
A'{R). Let {un,Vn) £ AA' be a minimizing sequence of A'. We may assume that 
Un,Vn £ fTo(B(0,i?„)) for some i?„ > 0. Therefore, {un,Vn) £ AT'^Rn) and 

A' = lim I{un,Vn) > lim A'{Rn) = A'{R), 


which completes the proof of the claim. 

Recalling (I4.41 i and (I4.71 i. for every {u, v) £ there exists > 0 with —>■ 1 

as e —>• 0 such that £ Af^. Then, 

limsup Ae < limsup/E(-^^u, = I{u,v), V(u,m) £ A/''(l). 

€—^0 £—^0 

It follows from (14.1 lb that 

lim sup Ag < ^'( 1 ) = A'. (-4 12) 

£->■0 ' ■ ^ 
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According to Proposition 14.11 we may let (ug, z;^) be a positive least energy solution 
of (14.6b . which is radially symmetric decreasing. By (14.7b and Sobolev inequality, we 
have 


A,= 


p*-2e-2 

2 (p* — 2£) Jb{o,i) 


[ (iVzter + iVu.r) > c > 0, V£ e (0, ^ 

7b(o,i) ^ 


(4.13) 

where C is independent of e. Then, it follows from (14.12b that Ue,Ve are uniformly 
bounded in iTg (B(0,1)). We may assume that uo,Ve vq, up to a subse¬ 

quence, weakly in iTg [B{0, 1)). Hence, {uq, vq) is a solution of 


(—lS.pU = pLi\u\^ xG 5(0,1), 

<-ApU =xG5(0, 1), (4.14) 

[zx,z;GiTi (5(0,1)). 

Suppose by contradiction that ||Me||oo + llwlloo is uniformly bounded. Then, by the 
Dominated Convergent Theorem, we get that 


lim / 

^“*■0 73(0,1) 


7s(0,l) 


lim / 


lim / 
^^°Jb{o.i) 


s(o.i) 


UqVo- 


/ 

7b(0,1) 


Combining these with I'^{us,Ve) = I'{uo,vo), similarly as the proof of Proposition 
14.11 we see that Ug —>■ —>■ vq strongly in Hq(^B{0, 1)). It follows from (14.13b 

that {uq, Vq) ^ (0, 0), and moreover, uq > 0,vo > 0. Without loss of generality, we 
may assume that uq ^ 0. By the strong maximum principle, we obtain that mq > 0 in 
5(0,1). By Pohozaev identity, we have a contradiction 


0< / (|Vuo|^ + iVvoT) (a; • B)(icr = 0, 

793(0,1) 


whereizis the outward unit normal vector on 95(0,1). Hence, HzteHoo + Hfelloo —>■ oo, 
as e —0. Let := max{uE(0), Ue(0)}. Since Ue{0) = maxs(o^i) Ue(a;) and 
Ue(0) = maxB(o 1 ) Ve{x), we see that Kg —-boo, as e: —>• 0. Setting 


Ue{x) := Ue{K^ “'x), Ve{x) := Ve{Kg. °'^x) 


p* — p — pe 
P 


we have 


max{5e(0),K(0)} 


= max < max Ug{x), max 

1 a:GB(0,if“=) a:eS(0,iV“') 



= 1 


(4.15) 


and {Ug,Vg) is a solution of 


-^pUg = 

-ApL, = 


I (q—£)7 jjci—i—e\rQ—e 
p*-2e e ’ 

(PpMjja-eyP-l-e 
p* — 2e ^ ^ ’ 


X G 5(0,A:“7, 
X G 5(0,A:“7. 
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Since 


\yU,{x)\Pdx = [ \VuMfdy 

Jrn 

= f |Vue(a;)|^dx < I |Vue(a;)|^da:, 

JR-W Jrn 


we see that {(C4, T4)}n>i is bounded in D. By elliptic estimates, we get that, up 
to a subsequence, (C4,14) —?► {U,V) S D uniformly in every compact subset of 
as e —?► 0, and (U,V) is a solution of (II.lb . that is, I'(U,V) = 0. Moreover, 
17 > 0, y > 0 are radially symmetric decreasing. By (14.15b . we have ([/, V) ^ (0,0) 
and so ([/, V) £ Af'. Thus, 


71' < nu, F) = (i -1) / (ivc/r + \vv\p)d 

\v V / .hiN 


\p p 




P ^ JB(0,Ktn 


< 


liminffi- [ (IV17JP +|VFJ^’)dx 

e^O \p p* — 2e) 

liminffi- f (I + I Vue|^)dx 

e^o Vp p* — 2eJ Jb(o.i) 


= lim inf . 

e^O 


It follows from (14.12b that A' < I{U, V) < liminf£_i.o A^ < N, which means that 
/([/, 17) = A!. By (14.9b . we get that U ^ 0 and 17 ^ 0. The strong maximum principle 
guarantees that U > 0 and 17 > 0. Since {U, 17) £ Af, we have I{U, 17) > 2l > AA. 
Therefore, 

I{U,V)= A = A', (4.16) 

that is, ([/, 17) is a positive least energy solution of (II. lb with {Hi) holding, which is 
radially symmetric decreasing. This completes the proof. □ 

Remark 4.1. If {Hi) and {C 2 ) hold, then it can be seen from Theorems U .2\ and \l .3\ 
that {f% Ue,y, f /ToUe,y) is a positive least energy solution of ( 17.7b . where (fco, lo) *7 
defined by ( li.iOb and Ue,y is defined by MAh 

Proof of Theorem ll.4l To prove the existence of (^(7), l{y)) for 7 > 0 small, recall- 

ing dljll, we denote Ffik, 1) by Fi{k, /, 7), * = 1, 2 in this proof. Let fc(0) = Pi 

and Z(0) = P 2 Then Fi(fc(0),/(O), 0 ) = 7^2 (^(0), ^0), O) = 0. Obviously, we 
have 

dkFi{k{0),l{0),0) = Pl^pik^^ > 0, 
aiFi(fc(0),Z(0),0) =5fci^2(fc(0),Z(0),0) =0, 
diF 2 {k{Q),l{ 0 ), 0 ) = > 0 , 
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which implies that 


( dkFUk{0),l{0),0) diFUk{0)Ji0),0) 

V dkF2{k{0)J{0),0) diF2(k(0),l(0),0) 


> 0 . 


By the implicit function theorem, we see that k(^), l{'y) are well defined and of class 
in (- 72 , 72 ) for some 72 > 0 , and (^( 7 ), ^( 7 ), 7 ) = ^ 2 (^( 7 ), ^( 7 ), 7 ) = 0 . 
Then, {^k{-/)U,^y, 0^Ue,y) is a positive solution of (fTTT) . Noticing that 

lim (fc( 7 ) + Z( 7 )) =A:(0) + ;(0) =pi ” +^2 " ^ 


there exists 71 £ ( 0 , 72 ] such that 

k{"/)+ l{j) > min {fi-^ ” > ^^2 ” 1 ^ V7e(0,7i). 

It follows from (I4.91 l and (14.161) that 

l{^/WF)U,,y, VWf)Ue,y) = ^(fc(7) + ^(7))5^ 

2. _Njz£ ^ 1 _N^ 

> minj—/Ti " *5” ^ 5 'p I 

> A' = A = I{U,V), 


that is, when (Ffi) is satisfied, {^k(^)Ue,y^ ^K'l)Ue,y) is a different positive solu¬ 
tion of (II.lb with respect to (f/, 17). This completes the proof. □ 


5 Proof of Theorem 11.51 


In this section, we consider the case (i 72 )- 
Proposition 5.1. Let 9 , r > 1 satisfy q + r < p* and set 


Sq,r{^) 


Sq+r{^) 


/^(|Vu|P + |Vt;|^)dx 
|y|g |u|r (jx) 


,xGwg(n) {J^\u\<i+^ dx)^^"' 


Then 

Sq^m = Sq+rm. (5.1) 

(q? r’’) 1+’' 


Moreover, if Uq is a minimizer for Sq+r{^), then (qp Uo,rr Uq) is a minimizer for 
Sq,r{^). 
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Proof. For u f 0 in Wg^’^(ri) and t > 0, taking v = t p u in the first quotient gives 


Sq,r{^) < 


f 9+p + t 1+’- 


{J^\u\^+rdx) 


P 5 
q+r 


and minimizing the right-hand side over u and t shows that Sq^ri^) is less than or 
equal to the right-hand side of (15.1b . For u, u 7 ^ 0 in WQ^’^(f2), let w = fp v, where 

^3±f ^ In 1^1'^"^'' dx 

J^\v\<i+-dx- 

Then dx = dx and hence 


/ \u\'^ \wf dx < / |u|‘^+’’da;= / dx 


by the Holder inequality, so 

/^(|Vu|P + |Vu|P)dx 
' dx)' 


(^|u|9|t;rdx)''+'- 


In |Vu|P -I- t ■>+'■ |Vr(;|P^ dx 
(f^lul^lwfdx)^ 

r folVul^dx <2 folVtul^dx 

> ti+7 ---_+i-i+F 




(In 


\w 


1 +’’ dx) “+ 


> 


tl + r -\-t l + r 


'q+r 


(H). 


The last expression is greater than or equal to the right-hand side of ( 15.1b . so minimiz¬ 
ing over {u, v) gives the reverse inequality. □ 


By Pror)osition l5.1l 


Sa,bm = Ai(0), S, 

(a“/3d)F^ 


(5.2) 


where Ai (H) > 0 is the first Dirichlet eigenvalue of — Ap in H. When {H 2 ) is satisfied, 
we will obtain a nontrivial nonnegative solution of system (O for A < Sa,b{n). 
Consider the (7^-functional 

$(tu) = - / r|Vu|^-f |Vu|^ — A(m^)“(u ''')^1 dx—— / (u''')“(u''')^dx, w G IF, 
pJn P* Jn 

where W = Z?g’^(r2) x Dg’^(S7) with the norm given by ||w||^ = |Vu|p -I- |Vu|^ for 
w = {u,v), I • Ip denotes the norm in U'{Vi), and m^(x) = max{±u(x),0} are the 
positive and negative parts of u, respectively. If w is a critical point of $, 

0 = T>'H(M-,t;-) = [ {\yu-\P + \yv-\P)dx 

Jn 
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and hence {u ,v ) = 0, so tc = («■*■,?;+) is a nonnegative weak solution of (fTTl i 
with {H 2 ) holding. 

K 
S ^ 

Proposition 5.2. IfO 7^ c < and A < Sa,b{^), then every {PS)c sequence o/$ 
has a subsequence that converges weakly to a nontrivial critical point 0/$. 

Proof. Let {wj} be a {PS)c sequence. Then 

=- / [|VMir + |Vuj|P-A(M+)“(v+)''] dx-^ f {up°‘{v+)^dx 

P JQ P Jq, 

=c+ 0(1) 


and 


^'{w,)w,= f [\yufP + \yvfP 
=o{\H\\), 


x{u+r{v+)»]dx 


so 


/ {u+r ivtfdx 

JQ. 

(5.3) 


( [|Vuj|P + |Vuj|P-A(u+)“(u+)^]dj: = c + o(||wj|| + l). (5.4) 

j r2 

Since the integral on the left is greater than or equal to (1 — 5 ^(o) ^ 

Sa.bifl), and p > 1, it follows that {wj} is bounded in W. So a renamed subsequence 
converges to some w weakly in W, strongly in L®(n) x L*{yi) for all 1 < s, f < p*, 
and a.e. in fl. Then wj —>■ w strongly in VLq x VLq for all 1 < g, r < p by 
Boccardo and Murat |l6l Theorem 2.1], and hence VtUj —7 Vtu a.e. in fl for a further 
subsequence. It then follows that in is a critical point of $. 

Suppose w = 0. Since {tu^j is bounded in W and converges to zero in LP[V>.) x 
LP{il), (I5.3I 1 and the Holder inequality gives 


0(1) = ^ (|vu,r + \yv,ndx - J^{u+r (v+fdx > ntu.r (i 


If llruj II 0, then ^(wj) —t 0, contradicting c 7^ 0, so this implies 


Ikjir > + 0(1) 

for a renamed subsequence. Then (15.4b gives 


J1 

c = h 0(1) > —^ + 0(1), 


N 



M. 

S ^ 

contradicting c < . 


□ 
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Recalling (11.41) and (11.5b . let rj : [0, oo) [0,1] be a smooth cut-off function such 
that r]{s) = 1 for s < and r]{s) = 0 for s > ^, and set 


Ue,p{x) = Uefi{x) 

for p > 0. We have the following estimates for (see ifTSl Lemma 3.1]): 


/r" 


IVUeJ^dx <ST +c{- 


/R'V 


rfa; > <1 ^ 
C 




eP -CpP 


-CeP, N=p‘^, 

, N>p'^, 


N-p 

e \ 


/ 


ulpdx>ST - C \ - 


N 

e \ 


(5.5) 

(5.6) 


(5.7) 


where C = C{N,p). We will make use of these estimates in the proof of our last 
theorem. 

Proof of Theorem ll.51 In view of ( 15.2b . 


$(tu) > - f 1 - 

P\ Sa,b{^)J 

P ^a ,/3 

SO the origin is a strict local minimizer of $. We may assume without loss of gen¬ 
erality that 0 € n. Fix p > 0 so small that fl D Bp{0) D suppue,p, and let 
We = (ap Ue^p,Pp Ue,p) G W. Noting that 

‘^{RWe) = ^ (p*\Vue,p\P - - ^ap Pp\Ue,p\p, 

—>■ —c» 


as i? —fix i?o > 0 so large that ^{RoWe) < 0. Then let 

r = {7 G C([0,1], fL) : 7(0) = 0, 7(1) = 7?ow;J 


and set 


c := inf max $(7(f)) > 0. 

7 er te[o.i] 

By the mountain pass theorem, $ has a {PS)c sequence {tu^ }. 
Since 1 1—>■ tRoWe is a path in F, 


c < max ^(tRoWe) 
~ te[o.i] 


1 

N 


'p*\Vue,p\P-Xia-p'^)p' 


^e,p\p 




p 

e.Plp- 



(5.8) 
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By (ES-diTli, 


Xicy^ 8^) p 

p* SP + ^ ^ ’ eP log e + 0(eP) 

^e<-^^ 

{a°‘8^)^ (^Sp + 0{ep^)j 


— ^a,0 


' “__s_ fi 

Xap p*/3p 

c^p-i 


|loge| + 0(1) eP 


if N = and 


<- 


p* S p — 


jv. A(a“/3^)p 


N-p ^ 


C 


eP + 0{e p-i ) 


1 / N N \ ' 

{a°‘8^)^ \ S~ + 0{e~)] 


—*^a,/3 


Xap ^/3p ^ 

N — p 

cs— 


+ 0(e- 


K 

g p 

if N > p^, so S'e < Sa,p if £ > 0 is sufficiently small. So c < by dS.Sl l. 
and hence a subsequence of {wj } converges weakly to a nontrivial critical point of $ 
by Proposition 15.21 which then is a nontrivial nonnegative solution of (II.Il l with {H 2 ) 
holding. □ 
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